Television programming is becoming more widely available in 3D. Sporting events and concerts have been broadcast for home consumption. As 3D component sales ramp up and as the demand for 3D grows, it is expected that 3D programming will be offered widely on most of the popular TV channels in the near future.
Viewers currently switch channels quite frequently among various programming content. Since this content is all typically two-dimensional (2D), there is usually no viewing difficulty or discomfort as content is switched by a viewer or for a viewer by the service provider. As the amount of viewable 3D programming content becomes more widely available in the future, the viewer is likely to encounter viewing problems when switching between channels, especially when one or more of these channels involves 3D programming content.
One problem in viewing 3D content occurs when adjacent image frame pairs have significant difference in depth. This is more commonly known as a “depth jump” because the depth at the regions of interest is perceived to change significantly between these two image frame pairs. When a viewer is channel surfing (i.e., changing channels in relatively quick succession), depth jumps may occur in many different situations. Depth for 3D content on one channel may be different from the depth of the 3D content on a newly tuned channel. When advertising content is inserted into the 3D programming content, the advertising content may exhibit a different depth, or even a different dimensionality from the 3D programming content. Dimensionality can change from 3D to 2D when the scene depth changes to zero. Within the same 3D programming content, depth jumps tend to occur concurrent with a shot or scene change. When depth jumps occur, viewers can experience varying degrees of discomfort.
Depth jumps within programming content such as a 3D movie are generally avoided or mitigated in the production phase or post-production phases. Unfortunately, when these problems are caused either by the viewer, when changing programs, or by the service providers, when inserting additional 2D or 3D content (e.g., advertisements), depth jumps cannot be avoided because they occur in real time and in an unplanned manner.
When time and circumstances permit, it is obvious that an opportune stage in which to avoid or even remove a depth jump is the production stage. In the production stage, there is full control over the content without any real-time constraints. During 3D content production, it is somewhat easier to remove the depth jump since the producer has full control over the presentation of the content. For example, when 3D content is being generated by computer graphics, the frames may be edited and rendered until there are no apparent depth jumps in the scene sequence.
Even when dealing with live shot films, the appearance of a depth jump between shots can be ameliorated by using well known techniques such as a cross-fade effect or an active depth cut. But these techniques have limited use for the real time scenarios confronting the 3D TV viewer. For example, although the cross-fade effect has been extensively used in producing audio effects and is easily implemented for video, its use would be inappropriate for all instances of shot switching, especially when a clear picture quality rather than a faded one is required during the transition period between shots.
In generating active depth cuts, outgoing and incoming shots are re-converged toward each other over some specified number of frames. Active depth cuts require implementation of a manual image shifting technique in the post-production stage. It is time consuming and it increases cost.
In the 3D TV environment, the luxury of production and post-production operations is not afforded due to the real-time nature of the depth jumps, either the ones that are viewer initiated via channel surfing or those that are initiated by the service provider via advertising insertion and the like. Depth jumps should be corrected in real time at the broadcast end or at the viewer end, such as in either a set top box (STB) or the TV itself, in order to reduce the potential for viewer discomfort.
For 3D TV broadcasting, it has been explained above that commercials are often inserted into a TV broadcast program as the program content is being broadcast. It does not appear to be practical to adjust the depth at the service provider location (e.g., such as a cable or satellite service provider or distributor) using active depth cut or cross-fade methods every time a commercial is inserted into programming content. Different commercials may be inserted into the same programming content for broadcast to different geographic audience.
None of the techniques discussed above or known in the art are suitable for dealing with these depth jumps to avoid wholly, or lessen partially, any possible viewer discomfort. Hence, an automatic method to adjust the depth of 3D content in real time, such as during the actual broadcast of the 3D content, appears to be needed in order to enhance the viewing experience of the 3D content.